Papers with unseen task
Multi-Task Transfer Matters During Instruction-Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Instruction-tuning improves a model’s ability to learn in-context, but the mechanisms that drive in-constext learning are poorly understood. |
| Approach: | They propose to train a model on hundreds of tasks to improve its ability to learn in-context. |
| Outcome: | The proposed methods improve model transfer and in-context generalization, suggesting catastrophic forgetting may impact in-constext learning. |
The Good, the Bad, and the Debatable: A Survey on the Impacts of Data for In-Context Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) is an emergent capability of large language models that allows them to learn new tasks at inference time without updating parameter updates. |
| Approach: | They propose to examine the relationship between data and in-context learning by examining the qualities of demonstrations that are desirable when selecting demonstrations, the "bad" qualities of demonstrators that can negatively impact the model and the "debatable" qualities. |
| Outcome: | The proposed model can learn unseen tasks by seeing a number of examples in the context window without updating parameters. |